Configuring: Moving Statistics

Define how to determine the calculation window for the statistics, and select one or more fields - the values of which will be used for the calculations. The resulting dataset will only contain the statistics of the selected fields, and none of the original values.

Worked example: Moving statistics

Steps to configure

  1. Configure calculation: Select either an index or time base to determine the calculation window, and the percentage of good quality data to be used.

  2. Select fields: Select the fields to be used.

  3. Select statistics: Select the type of statistics to be calculated.

  4. Execute: A new dataset is created, containing the selected calculated statistics.

Step 1: Configure calculation

Select how to determine the time window to be used. This is the window over which the statistics will be calculated. Also specify the good quality threshold for your statistics.

Index based:

  • Enter the number of rows to be used as the moving window over which selected statistics will be calculated.

  • Default: index based is selected by default.

  • Minimum configuration: the user must enter greater than 0 rows.

Time based:

  • Enter the time window in seconds.

  • If you have more than one timestamp field, select the required field to be used from the drop down list of all available timestamp fields.

  • Default: blank textbox for number of seconds, and the first available timestamp field listed.

  • Minimum configuration: there has to be at least one timestamp field.

Good quality threshold:

 

  • A good quality between 0 and 100 has to be entered.

  • This percentage is a standard requiring a certain percentage of the fields in the time window to be of good quality.

  • The values in each window need to adhere to the quality percentage in order to give the specific statistic in the resulting dataset.

  • For example: For a dataset with 100 rows, and you select an index based time span of 10 and a good quality percentage of 50%: In order for this operation to calculate a value for the statistic, 5 out of the 10 rows in the window need to have good quality values.

  • Apply quality threshold using the full data window:

    • This is the exact size of the data window specified above.

    • For example: if the data window is 10 rows, and a good quality threshold of 60% is specified, then at least 6 out of the 10 rows must be good quality data in order for the statistics to be calculated. If there are not 6 out of 10 rows of good quality data, the row window will move until there are 6 out of 10 rows of good quality data.

  • Apply quality threshold using partial data window:

    • This window can use a partial number of rows of good quality data until the exact size of the data window specified above is reached.

    • For example: if the data window is 10 rows, and a good quality threshold of 60% is specified, then at least 60% of the row data must be good quality data in order for the statistics to be calculated.

    • This means that if there is only row of data, of good quality, then the quality percentage is 1/1 = 100%. This is above the 60% threshold, and therefore will be used for calculating the statistics.

    • If in the first two rows of data, only one of these rows has good data, then the quality percentage is 1/2 = 50%. This is below the 60% threshold, and therefore will not be used for calculating the statistics.

  • These examples can be interpreted the same way for a time based window.

Step 2: Select fields

Select the fields to be used when calculating the statistics.

  • This is an optional step.

  • Default: all the fields are selected.

  • Minimum configuration: at least one field has to be selected.

Step 3: Select statistics

Select the statistics to be calculated for the selected fields.

  • Average: the sum of all the values in the window, divided by the count of values.

  • Kurtosis: a measure of whether the data are peaked or flat relative to a normal distribution. Data sets with high kurtosis tend to have a distinct peak near the mean, decline rather rapidly, and have heavy tails. Data sets with low kurtosis tend to have a flat top near the mean rather than a sharp peak.

  • Maximum:the greatest value in the window.

  • Minimum: the smallest value in the window.

  • Skew: a measure of the asymmetry of the distribution of data values. For example, in a histogram, skew occurs if the values on one side of the histogram tend to extend further from the median than the values on the other side.

  • Standard deviation: measure of statistical dispersion; it is calculated as the positive square root of the variance.

  • Variance: measure of statistical dispersion, averaging the squared distance of its possible values from the mean.  

  • Default: all the statistics are selected by default.

  • Minimum configuration: at least one statistic has to be selected.

Step 4: Execute

A new dataset will be created, containing only the statistics for the selected fields. The original dataset is not affected in any way.


Related topics:

  

CSense 2023- Last updated: June 24,2025